Skip to content

ci: run PR title lint on release-please bot PRs - #80

Draft
nicklasl wants to merge 9 commits into
mainfrom
nicklasl/fix-lint-pr-bot-trigger
Draft

nicklasl wants to merge 9 commits into
mainfrom
nicklasl/fix-lint-pr-bot-trigger

Conversation

@nicklasl

@nicklasl nicklasl commented Nov 2, 2026 •

Copy link
Copy Markdown
Member

Problem

Lint PR triggers on pull_request_target (opened/edited/synchronize). The release-please release PR (branch release-please--branches--main--components--confidence_flutter_sdk, most recently #71) is created and force-pushed by the release-please action using the workflow's default GITHUB_TOKEN — and those bot-caused events never trigger pull_request_target workflows at all. So Validate PR title never appears on release PRs unless a human happens to touch the PR (e.g. run 41417126839 was summoned only by a human body edit on #71, an hour after the bot's fold-in).

Root cause (verified from run history, with a twist)

Every one of the 50 runs ever recorded on the release branch fits one pattern: every Lint PR run is human-actor (4× vahidlazio 2025-09-22, 1× nicklasl 2026-11-02); bot pushes never produced a single pull_request_target run. Meanwhile the pull_request workflows (Test-Android/CI/Test-iOS) do get runs created on bot events — every release-please PR opening and every fold-in since June 2026.

The twist (found by testing): bot-actor pull_request runs are created action_required and only execute after a maintainer approves them. Run history confirms this has always been true: e.g. the Nov 2 fold-in runs (CI 41410137582) were created 08:02:04 as attempt 1 action_required, and ran as attempt 2 only after approval at 08:02:15 — the same release-prep approval that made the flutter checks green on #71 before it merged. This approval gesture is already part of the release routine; approving the PR's workflows is the only way the flutter checks ever ran on a release PR.

Fix

Switch the trigger from pull_request_target to pull_request (same event types) and make the two sticky-comment steps continue-on-error: true:

  • Bot path: pull_request runs are created on bot opened + every synchronize → Validate PR title shows up on release PRs, and executes (and validates) with the same workflow approval maintainers already give the flutter checks during release prep. Today it never appears at all.
  • Human same-repo PRs: unchanged — the check fires and comments work (the e2e test below also proves the repo-default token can post sticky comments).
  • Fork PRs: the pull_request token is read-only there, so commenting degrades gracefully (continue-on-error) but title validation still runs and the check still reports pass/fail.
  • Deliberately no explicit permissions: block (see comment in the file): the workflow needs nothing beyond the repo-default token, and explicit blocks add no value here.

Validate PR title stays a non-required check on purpose: a required check that can silently not fire would block release PRs forever.

Verification

  1. Human path (this PR): opened by a human with the modified workflow on its head branch → Validate PR title is green here (you see two of them while this PR is open: the old pull_request_target copy from main plus the new pull_request one; post-merge only one remains). Historical human-PR green runs (feat: replace native SDK bridge with Dart #62/fix!: reply exactly once, on the platform thread, from async channel cases #76/feat!: replace Flutter bridge with Dart OpenFeature provider #79 style) are untouched by the change.
  2. Bot path (e2e test, full sequence reproduced): a temporary workflow_dispatch/push-triggered workflow (now removed from this PR) reproduced release-please's exact conditions: created a throwaway branch, opened a PR to main's content via GITHUB_TOKEN (bot opened), then pushed a commit onto the PR branch via GITHUB_TOKEN's git data API (bot synchronize, same mechanics release-please uses), with a deliberately invalid title. It asserted the run is created, awaited the maintainer approval (the release-prep gesture), and verified the job — not just the run — failed with the invalid title, plus the sticky failure comment posting. Result — all green:
    • Test run: 41422611248 (success), throwaway PR throwaway bot-path test 4 with invalid title #84 (closed & cleaned up)
    • Bot opened → Lint PR run 41422639137: Validate PR title job failure on the invalid title ✅
    • Bot synchronize (fold-in) → Lint PR run 41422744814: Validate PR title job failure ✅
    • Sticky failure comment posted on the bot PR: yes ✅ (also proves the sticky-comment step works with the repo-default token)
  3. Definitive real-world proof: the next release-please fold-in on main after this merges — Validate PR title should appear on the release PR (created immediately; validating once the usual workflow approval is given), with no human touching the PR.

Honest limits

  • A human-pushed PR (like this one) fires both trigger styles, so it cannot prove the bot path; only (2) and (3) can. The e2e test reproduced the approval step, but the next real fold-in remains the ground truth.
  • Bot-actor pull_request runs (including this check on release PRs) are created pending approval and execute only after the release-prep workflow approval maintainers already perform for the flutter checks. Between a fold-in and that approval, the check shows as awaiting approval rather than validated. This is a property of GitHub's approval rules for non-collaborator actors, not of this change — the only trigger that skips it entirely (pull_request_target) never fires on bot events at all.
  • Release-please titles are conventional-valid by construction (all four historical human-summoned lint runs on release-PR titles were green), so the expected steady-state on a release PR is a green Validate PR title after approval — this is about the check finally existing at all.

Note on the android-test check on this PR's final commit: it flaked three consecutive times on pure emulator infrastructure (a hung run android tests step cancelled at 45+ min, then an adb ... exit code 1, then a flutter-test websocket HttpException: Connection closed before full header was received — run 41423300942, attempts 1–3). The identical tree passed android-test green on the previous commit of this branch (41422937654) and on main this morning (PR #71, after the usual single flake rerun) — this diff touches only .github/workflows/lint-pr.yaml and cannot affect the Android build. Treat the red android-test as the repo's known emulator roulette; a gh run rerun --failed at review time should clear it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant